{ "cells": [ { "cell_type": "markdown", "id": "65b25242-6c89-4584-8a7f-6f0c35fdbba7", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "# ‘Farming’ with ``Harvester`` Example\n", "\n", "This examples demonstrates ‘farming’ runs, that is, using a {class}`~xyzpy.Harvester` to perform and combine succesive, disjoint sets of runs.\n", "\n", "First we define a simple trig function to explore and label with a {class}`~xyzpy.Runner`:" ] }, { "cell_type": "code", "execution_count": 1, "id": "216843ba-6d60-4865-8e9a-7d26cedbb125", "metadata": {}, "outputs": [], "source": [ "%config InlineBackend.figure_formats = ['svg']\n", "import numpy as np\n", "\n", "import xyzpy as xyz\n", "\n", "\n", "def trig(f, x, amp, phi):\n", "\n", " trig_fn = getattr(np, f)\n", " err = 0.2 + 0.1 * np.random.randn()\n", "\n", " return amp * trig_fn(x - phi), err\n", "\n", "\n", "r = xyz.Runner(trig, [\"f(x)\", \"ef(x)\"])" ] }, { "cell_type": "markdown", "id": "a0aa032d-eb53-4396-8413-b91e30543a14", "metadata": {}, "source": [ "The harvester is composed of this ``Runner`` and logic for merging for saving each new set of runs." ] }, { "cell_type": "code", "execution_count": 2, "id": "cd44dd71-2149-4fb3-b409-2f9e82c8478d", "metadata": {}, "outputs": [], "source": [ "h = xyz.Harvester(r, data_name=\"trig.h5\")" ] }, { "cell_type": "markdown", "id": "cd4b4389-5629-4030-8cb5-ec245047193e", "metadata": {}, "source": [ "We perform one set of runs first:" ] }, { "cell_type": "code", "execution_count": 3, "id": "55761ed1-9d3a-4e96-b262-bd9c30316648", "metadata": {}, "outputs": [], "source": [ "combos_1 = {\n", " \"f\": [\"sin\", \"cos\"],\n", " \"x\": np.linspace(-5, 5, 101),\n", " \"amp\": [1, 2, 3],\n", " \"phi\": [0.4, 0.8],\n", "}" ] }, { "cell_type": "code", "execution_count": 4, "id": "9d542fdb-bbd4-4e6d-bff1-b33439192819", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|##########| 1212/1212 [00:00<00:00, 871954.79it/s]\n" ] } ], "source": [ "h.harvest_combos(combos_1)" ] }, { "cell_type": "markdown", "id": "417a469e-d24f-4844-a681-3519f1c7c8de", "metadata": {}, "source": [ "Which we can plot:" ] }, { "cell_type": "code", "execution_count": 5, "id": "06e74169-1726-42bf-81ac-d8bb3651c991", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "" ], "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig, axs = h.full_ds.xyz.plot(\n", " x=\"x\",\n", " y=\"f(x)\",\n", " color=\"f\",\n", " col=\"amp\",\n", " row=\"phi\",\n", " err=\"ef(x)\",\n", ")" ] }, { "cell_type": "markdown", "id": "5dec0cd4-a8f9-4536-96c8-36c84f42b799", "metadata": {}, "source": [ "Then we can define and run a completely disjoint set of runs:" ] }, { "cell_type": "code", "execution_count": 6, "id": "86bbc3e2-2226-4404-ac17-a3c4091581ad", "metadata": {}, "outputs": [], "source": [ "combos_2 = {\n", " \"f\": [\"sin\", \"cos\"],\n", " \"x\": np.linspace(-5, 5, 101),\n", " \"amp\": [4, 5],\n", " \"phi\": [1.2, 1.6],\n", "}" ] }, { "cell_type": "code", "execution_count": 7, "id": "d67012b5-691b-421a-8600-53c23bb2281a", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|##########| 808/808 [00:00<00:00, 283883.20it/s]\n" ] } ], "source": [ "h.harvest_combos(combos_2)" ] }, { "cell_type": "markdown", "id": "eb9ae649-a779-4cf9-bcd5-cbc04cf7b0f3", "metadata": {}, "source": [ "If we plot the current ``full_ds`` we’ll see both sets of runs are present, but there are also blank spaces:" ] }, { "cell_type": "code", "execution_count": 8, "id": "33b7fbd0-a710-4e61-b412-7b43db5d51ad", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "" ], "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig, axs = h.full_ds.xyz.plot(\n", " x=\"x\",\n", " y=\"f(x)\",\n", " color=\"f\",\n", " col=\"amp\",\n", " row=\"phi\",\n", " err=\"ef(x)\",\n", ")" ] }, { "cell_type": "markdown", "id": "2759a756-fe57-418e-b292-db2175af88a3", "metadata": {}, "source": [ "Finally let’s add ``tan`` into the mix, but only for the central combinations:" ] }, { "cell_type": "code", "execution_count": 9, "id": "5775a437-e7d0-41c3-a640-310ac10b0428", "metadata": {}, "outputs": [], "source": [ "combos_3 = {\n", " \"f\": [\"tan\"],\n", " \"x\": np.linspace(-5, 5, 101),\n", " \"amp\": [2, 3, 4],\n", " \"phi\": [0.8, 1.2],\n", "}" ] }, { "cell_type": "code", "execution_count": 10, "id": "4d3fa0d3-82c5-4edc-87eb-699160807410", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|##########| 606/606 [00:00<00:00, 1130168.17it/s]\n" ] } ], "source": [ "h.harvest_combos(combos_3)" ] }, { "cell_type": "markdown", "id": "1f6dc6a4-cd6f-4711-a2f5-144d34e5d39c", "metadata": {}, "source": [ "We’ll specify some plotting limits to avoid the infinities as well:" ] }, { "cell_type": "code", "execution_count": 11, "id": "22313324-501e-43f3-83b6-55291a629cf2", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "" ], "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig, axs = h.full_ds.xyz.plot(\n", " x=\"x\",\n", " y=\"f(x)\",\n", " color=\"f\",\n", " col=\"amp\",\n", " row=\"phi\",\n", " err=\"ef(x)\",\n", " ylim=(-4, 4),\n", ")" ] }, { "cell_type": "markdown", "id": "eb4c0b61-0d5e-4f73-ad56-e20a80955bc1", "metadata": {}, "source": [ "All runs have also been accumulated into the on-disk:" ] }, { "cell_type": "code", "execution_count": 12, "id": "732e95fa-3d75-4560-80ed-17924e36619d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "trig.h5\n" ] } ], "source": [ "!ls *.h5" ] }, { "cell_type": "code", "execution_count": 13, "id": "6c151369-d9ec-485c-8bb7-e90d1ff7ec3e", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset> Size: 98kB\n",
       "Dimensions:  (f: 3, x: 101, amp: 5, phi: 4)\n",
       "Coordinates:\n",
       "  * f        (f) <U3 36B 'cos' 'sin' 'tan'\n",
       "  * x        (x) float64 808B -5.0 -4.9 -4.8 -4.7 -4.6 ... 4.6 4.7 4.8 4.9 5.0\n",
       "  * amp      (amp) int64 40B 1 2 3 4 5\n",
       "  * phi      (phi) float64 32B 0.4 0.8 1.2 1.6\n",
       "Data variables:\n",
       "    f(x)     (f, x, amp, phi) float64 48kB 0.6347 0.8855 nan nan ... nan nan nan\n",
       "    ef(x)    (f, x, amp, phi) float64 48kB 0.08426 0.1342 nan ... nan nan nan
" ], "text/plain": [ " Size: 98kB\n", "Dimensions: (f: 3, x: 101, amp: 5, phi: 4)\n", "Coordinates:\n", " * f (f)